docs: clarify exact Jacobian contract inspection - #786
Merged
Conversation
Collaborator
Author
|
Follow-up paired evaluation (Terra, high reasoning, identical graph/topology prompts and budgets; two repetitions per task):
So the explicit CONTRACT guidance reduced MCP calls by 12.7%, elapsed time by 14.4%, and input tokens by 23.1%, but this small sample does not show a reliability improvement. The missing topology compositions were traced to the consumer contract rather than the instruction wording and are addressed separately in draft #805. I recommend treating #786 as an efficiency/clarity change, not claiming an assurance-rate gain from this sample. |
This was referenced Aug 8, 2026
morluto
marked this pull request as ready for review
August 8, 2026 17:34
morluto
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changed
math.find({capability_id, view: "CONTRACT"})form for inspecting a selected operation;mode: "CONTRACT"tomath.runor puttingCONTRACTinto a discovery query;Evaluation evidence
In the 24-run harder routing study, the Skill-guided agent found
poset.linear_extensions.countbut attempted the invalidmath.run(... mode: "CONTRACT")form and then searched for the wordCONTRACT. Similar inspection confusion contributed to excess graph turns.A focused forward-test with this patch used the correct exact inspection form. On the poset case it reduced MCP calls from 13 to 7 and input tokens from 304,743 to 168,176. The run still exposed a separate missing artifact-composition path for linear-extension counting; this PR does not claim to fix that product capability.
Overlap
PR #567 optimized common direct-run contracts and card metadata. It did not document the exact selected-capability inspection call or forbid the invalid forms observed here.
Validation
quick_validate.py: passedgit diff --check: passed